Comprehensiveness of tree based models: attribute dependencies and split selection

The attributes’ interdependencies have strong effect on understandability of tree based models. If strong dependencies between the attributes are not recognized and these attributes are not used as splits near the root of the tree this causes node replications in lower levels of the tree, blurs the description of dependencies and also might cause drop of accuracy. If Relief family of algorithms which is capable of estimating the attributes’ dependencies is used for split selectors we can partly overcome the problem. We describe ReliefF and RReliefF algorithms and their use in connection with tree based models. Some theoretical properties of Relief’s estimate and a recent empirical study suggest that accuracy optimization near the fringe of the tree is not necessary with these algorithms.